AWS Nitro Enclaveについて
軽い自己紹介
@ken5scal です
技術同人サークルやったり、本書いたりしてます
最近は Trust ってテーマにはまってます+
本業はアセマネです
AWS Nitro Enclave
色々な制約がついたCPUとメモリが隔離された仮想マシン on EC2
Enclave
飛び地、あるいは隔離
出島
in: Willis, David Blake; Murphy-Shigematsu, Stephen, eds. Transcultural Japan: at the borderlands of race, gender, and identity. London; New York: Routledge, 2008. xxv, 342p. (361/ 177): p. 239-263
ストレージなし、ネットワークアクセスなし、インタラクティブなし、メタデータなし、DNSなし、NTPなし
ユースケース
機密性が高い情報を処理したり
（Enclaveからの）連携先サービスのリクエスト元として真正であることを確認してもらいたい...といったユースケース
https://gyazo.com/8c85f1196e5a712ca3e5625a85022371
とりあえず動かしてみよう
kms tool: 隔離されたEnclave内でのみ秘匿化されたメッセージを複合する
https://gyazo.com/9102a3c66088e53bf319851d618a6102
code:bash
# kmstool-enclaveを走らせる
sh-4.2$ docker build --target kmstool-enclave -t kmstool-enclave -f containers/Dockerfile.al2 .
sh-4.2$ nitro-cli build-enclave --docker-uri kmstool-enclave --output-file kmstool.eif
sh-4.2$ sudo systemctl restart nitro-enclaves-allocator.service
sh-4.2$ nitro-cli run-enclave --eif-path kmstool.eif --memory 1024 --cpu-count 2 --debug-mode
# kmstool-instanceを走らせる
sh-4.2$ vsock-proxy 8000 kms.ap-northeast-1.amazonaws.com 443
sh-4.2$ export AWS_DEFAULT_REGION=ap-northeast-1
sh-4.2$ ENCLAVE_CID=$(nitro-cli describe-enclaves | jq -r .0.EnclaveCID) sh-4.2$ MESSAGE="Hello, KMS\!" ★★★
sh-4.2$ KMS_KEY_ARN=arn:aws:kms:ap-northeast-1:297323088823:key/8001b1d6-892d-446f-ad2a-6f8902dc9b5d
sh-4.2$ CIPHERTEXT=$(aws kms encrypt --key-id "$KMS_KEY_ARN" --plaintext "$MESSAGE" --query CiphertextBlob --output text)
sh-4.2$ docker run --network host -it kmstool-instance /kmstool_instance --cid "$ENCLAVE_CID" --region "$AWS_DEFAULT_REGION" "$CIPHERTEXT"
Object = { "Status": "Ok" }
Object = { "Operation": "Decrypt", "Ciphertext": "AQICAHjYtYgyOropB\/glEBRgDVA+1m\/mfZ1RRGH\/7e46K7MIvQH2Lri+GqGaPlWxYTmWbPItAAAAajBoBgkqhkiG9w0BBwagWzBZAgEAMFQGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMYm2dbBqunkkpjmbxAgEQgCezbsli0LPnz1\/QLr5Z06f\/AywAhPpULWugg5e2vly0QJ6\/OJOkrLw=" }
Object = { "Status": "Ok", "Message": "SGVsbG8sIEtNU1wh" }
Hello, KMS\! ★★★
見事に平文が帰りました....
だから何？
Attestation!
Attestationの語源 -> 契約における証明書。権威の裏付けが必要
連携先サービスのリクエスト元として真正であることを確認してもらいたい -> trust worsy entityであることを証明したい
CIの中で走らせる、そのコード、本当に自分らが作ったもの？
「その機密情報を扱うのがちゃんと隔離されて安全が物理的に保証されたワークロードでだけ使われてます」ということが証明できる
code:cloudtrail.json
{
"eventName": "Decrypt",
"awsRegion": "ap-northeast-1",
"sourceIPAddress": "54.168.188.252",
"requestParameters": {
"encryptionAlgorithm": "SYMMETRIC_DEFAULT"
},
"responseElements": null,
★★★"additionalEventData": {
"recipient": {
"attestationDocumentModuleId": "i-0736e34244dde3e74-enc0179c4df758b2837",
"attestationDocumentEnclaveImageDigest": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}
}
}
どう証明するか？
Enclaveのコードや構成をもって、trust worsy entityであることを証明。
構成情報はEnclave ImageFileを作成したときにNitro Hypervisorから提供される
code:run-enclave.sh
sh-4.2$ nitro-cli build-enclave --docker-uri kmstool-enclave --output-file kmstool.eif
Start building the Enclave Image...
Enclave Image successfully created.
(中略)
{
"Measurements": {
"HashAlgorithm": "Sha384 { ... }",
★★★"PCR0": "070f48e672535295eca71a5b9db1e6f4399b65a525e5a47ff302ecbff9a7d5aa1b0659e5b278797a3c421cd560f2e77e",
★★★"PCR1": "c35e620586e91ed40ca5ce360eedf77ba673719135951e293121cb3931220b00f87b5a15e94e25c01fecd08fc9139342",
★★★"PCR2": "245424d41b478e9b5ed42058823897cc372515092f9c881fd1c8c1aed46158e270670a529a0b4964e60fe14e5eb70627"
}
}
sh-4.2$ sudo systemctl restart nitro-enclaves-allocator.service
証明情報を構成することで、その証明情報の内容をもとにしたIAM Policyを記述可能
↓の例では、あるKMSのCMKを使って複合できる権限を特定Enclaveのみに限定
PCR0, PCR1, PCR2, PCR3, PCR4, PCR8が指定できて、PCR8だと実行したいアプリケーション（Docker, Enclave Image Fileレベル）で指定が可能
現時点ではKMSのDecrypt, GenerateDataKey, GenerateRandom Actionしか対応してない。
そのうちS3やSecret Managerなどのリソースポリシーにも適用されるかも？
本命はDBじゃないかな
code:kms-policy.json
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
(略)
{
"Sid": "Enable decrypt from enclave",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::297323088823:role/test-ec2-role"
},
"Action": "kms:Decrypt",
"Resource": "*",
"Condition": {
"StringEqualsIgnoreCase": {
"★★★kms:RecipientAttestation:ImageSha384": "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"★★★kms:RecipientAttestation:PCR1":"c35e620586e91ed40ca5ce360eedf77ba673719135951e293121cb3931220b00f87b5a15e94e25c01fecd08fc9139342"
}
}
},
(略)
]
}
enclaveが動くEC2インスタンスそのものから実行しても（KMSポリシーに）拒否られる
code:bash
sh-4.2$ aws kms decrypt --key-id $KMS_KEY_ARN --ciphertext-blob fileb:///tmp/cipher-text
# expected Behavior
An error occurred (AccessDeniedException) when calling the Decrypt operation: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
証明に裏付けを与える権威、つまり署名者はだれ？ -> AWS Nitro Attestation Public Key Infrastructure
Nitro Hypervisorは証明書(Attestation Document）を作る。
the enclave signing key,
a hash of the enclave image,
a hash of the parent instance ID,
a hash of the ARN of the attached IAM role
で、署名される
署名の仕組みは今後の課題に...
署名用の鍵がどのタイミングどこにロードされるのかがわかってない
code:bash
sh-4.2$ nitro-cli run-enclave --eif-path kmstool.eif --memory 1024 --cpu-count 2 --debug-mode
0.338140 Asymmetric key parser 'x509' registered (略)
0.376238 Key type dns_resolver registered 0.376828 NET: Registered protocol family 40 0.377867 sched_clock: Marking stable (376783151, 0)->(460833197, -84050046) 0.379384 Loading compiled-in X.509 certificates 0.380634 Loaded X.509 cert 'Build time autogenerated kernel key: c582bb2a1fdabdd195bbc9c8840fff8158852907' 他のプラットフォームでのAttestationと垂直統合
https://gyazo.com/08bc31ba9318a22faeff9f091ad51364
この仕組自体、別にAWSが独自ではない
世界的な流れを感じる
が、そもそもクライアントサイドでは結構昔からある。
TCG（Trusted Computing Group)によって、構成管理の証明や機微情報の安全な保管をするための技術仕様・標準化されてきた
Windows10: 2016年から搭載がマストになってるTPM2.0とかで実現されてる
歴史的経緯は↓にすごくまとまってる
言いたいことは、「クライアントサイドでもとからあった技術が、サーバーサイドに実装されていってるよ。それが意味することは一体なんだろう」ということ
同時にCPUからの垂直統合も流れになってるような気がする
例: Nitro System, Microsoft Pluton Processor, Apple M1, Google Whitechapel
（時間があれば）Windows10 のAttestationの仕組み
多分、Nitro Enclaveも同じイメージだがまだ調べきれてない
https://gyazo.com/0ff3cebe9564a4026d78c5609af5395c
https://gyazo.com/4aeb91049f5888ffffeccf2d8becffc0
https://gyazo.com/16cba77943e8a51674fcfceebcd1f7e6
https://gyazo.com/cbc38c0b4e4b41d7eeaed9c33da57645
よくわからん
セットアップがやや面倒くさい？
nitro-cli とかは、Enclaveを有効にするオプションをつけて立ち上げる時点でインストールされてて欲しい
が、そうでもないのか？ CIでenclave image file (efi) を作ってデプロイすればいいのか？
DevOpsの流れがまだ見てない。今後の調査課題
でも、Azure Confidential Computingも自分でセットしなきゃなので、業界全体の状態としてはまだまだヨチヨチあるきなのかも（勿論、GAされた時点で大きな進歩なのだが）
結局KMSポリシーに、そしてIAMポリシーに依拠するのでは？
VSOKCKで仲介するだけだが、その前提はKMSポリシー・IAMポリシーがやられない前提では？
SSFRやLambdaを使ったcompromiseでやられるのは最終的にはそこでは...?
どういう目的でNitro System作ってるか気になる
MicrosoftやGoogleはどっちかっていうと、Platformerとして顧客のデータを完全に秘密にしたい（Controllable）からというのを公言してるけど、あんまりAWSがそういうこと言ってないのは気になる
一方、spectreとか
code:bash
$sh nitro-cli run-enclave --eif-path kmstool.eif --memory 1024 --cpu-count 2 --debug-mode
0.008008 Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization 0.008894 Spectre V2 : Mitigation: Enhanced IBRS 0.009432 Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch 0.010359 Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier おしまい
https://gyazo.com/1e2dacff2de6ddc0987a3ff9488e15f9
ToDo
pcr8の作成
nitro attestation pkiの調査
冗長構成をもつenclave
attestationプロセスの調査
Ref
以下はメモです。
build enclave
code:dockerfile
FROM busybox
ENV HELLO="Hello from the enclave side!"
COPY hello.sh /bin/hello.sh
code:hello.sh
sh-4.2$ cat /usr/share/nitro_enclaves/examples/hello/hello.sh
# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
count=1
while true; do
printf "%4d $HELLO\n" $count count=$((count+1))
sleep 5
done
enclave file ( .eif)の作成..と measurement の作成
measurements are used to set up the attestation process
code:bash
sh-4.2$ docker build /usr/share/nitro_enclaves/examples/hello -t hello
sh-4.2$ sudo nitro-cli build-enclave --docker-uri hello:latest --output-file hello.eif
Start building the Enclave Image...
Enclave Image successfully created.{ "Measurements": {
"HashAlgorithm": "Sha384 { ... }",
"PCR0": "c23f62608bca532ca89537d8e5bdce115fa6279119395fbe1bff6ae1f6ec6882d9606d40d41d59c321631275dc1415ac",
"PCR1": "c35e620586e91ed40ca5ce360eedf77ba673719135951e293121cb3931220b00f87b5a15e94e25c01fecd08fc9139342", "PCR2": "1a7e38daac39aaa7acc3633b093403f28c8678af5efe3c3f84811794f17e67f6ec8b63bee726d0553289163051c4eb2a"
}
}
or...
code:bash
sh-4.2$ unzip main.zip
sh-4.2$ cd aws-nitro-enclaves-sdk-c-main/
sh-4.2$ docker build --target kmstool-enclave -t kmstool-enclave -f containers/Dockerfile.al2 .
sh-4.2$ docker build --target kmstool-instance -t kmstool-instance -f containers/Dockerfile.al2 .
sh-4.2$ nitro-cli build-enclave --docker-uri kmstool-enclave --output-file kmstool.eif
Start building the Enclave Image...
Enclave Image successfully created.
{
"Measurements": {
"HashAlgorithm": "Sha384 { ... }",
"PCR0": "070f48e672535295eca71a5b9db1e6f4399b65a525e5a47ff302ecbff9a7d5aa1b0659e5b278797a3c421cd560f2e77e",
"PCR1": "c35e620586e91ed40ca5ce360eedf77ba673719135951e293121cb3931220b00f87b5a15e94e25c01fecd08fc9139342",
"PCR2": "245424d41b478e9b5ed42058823897cc372515092f9c881fd1c8c1aed46158e270670a529a0b4964e60fe14e5eb70627"
}
}
sh-4.2$ sudo systemctl restart nitro-enclaves-allocator.service
run enclave
code:bash
sh-4.2$ nitro-cli run-enclave --eif-path kmstool.eif --memory 1024 --cpu-count 2 --debug-mode
Start allocating memory...
Started enclave with enclave-cid: 16, memory: 1024 MiB, cpu-ids: 1, 3 {
"EnclaveID": "i-0736e34244dde3e74-enc179c4df758b2837",
"ProcessID": 27881,
"EnclaveCID": 16,
"NumberOfCPUs": 2,
"CPUIDs": [
1,
3
],
"MemoryMiB": 1024
}
sh-4.2$ ENCLAVE_ID=$(nitro-cli describe-enclaves | jq -r .0.EnclaveID) sh-4.2$ nitro-cli console --enclave-id $ENCLAVE_ID
Connecting to the console for enclave 16...
Successfully connected to the console.
0.000000 Linux version 4.14.177-104.253.amzn2.x86_64 (mockbuild@ip-10-0-1-32) (gcc version 7.3.1 20180712 (Red Hat 7.3.1-6) (GCC)) #1 SMP Fri May 1 02:01:13 UTC 2020 0.000000 Command line: reboot=k panic=30 pci=off nomodules console=ttyS0 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd random.trust_cpu=on virtio_mmio.device=4K@0xd0000000:5 virtio_mmio.device=4K@0xd0001000:6 0.000000 x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' 0.000000 x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' 0.000000 x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' 0.000000 x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers' 0.000000 x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR' 0.000000 x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' 0.000000 x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' 0.000000 x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' 0.000000 x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' 0.000000 x86/fpu: xstate_offset2: 576, xstate_sizes2: 256 0.000000 x86/fpu: xstate_offset3: 832, xstate_sizes3: 64 0.000000 x86/fpu: xstate_offset4: 896, xstate_sizes4: 64 0.000000 x86/fpu: xstate_offset5: 960, xstate_sizes5: 64 0.000000 x86/fpu: xstate_offset6: 1024, xstate_sizes6: 512 0.000000 x86/fpu: xstate_offset7: 1536, xstate_sizes7: 1024 0.000000 x86/fpu: xstate_offset9: 2560, xstate_sizes9: 8 0.000000 x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format. 0.000000 e820: BIOS-provided physical RAM map: 0.000000 NX (Execute Disable) protection: active 0.000000 tsc: Fast TSC calibration using PIT 0.000000 e820: last_pfn = 0x3f800 max_arch_pfn = 0x400000000 0.000000 x86/PAT: MTRRs disabled, skipping PAT initialization too. 0.000000 CPU MTRRs all blank - virtualized system. 0.000000 x86/PAT: Configuration 0-7: WB WT UC- UC WB WT UC- UC 0.000000 Scanning 1 areas for low memory corruption 0.000000 Using GB pages for direct mapping 0.000000 kvm-clock: Using msrs 4b564d01 and 4b564d00 0.000000 kvm-clock: cpu 0, msr 0:3f7dc001, primary cpu clock 0.000000 kvm-clock: using sched offset of 1900088432 cycles 0.000000 clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns 0.000000 Movable zone start for each node 0.000000 Intel MultiProcessor Specification v1.4 0.000000 MPTABLE: Product ID: 000000000000 0.000000 IOAPIC0: apic_id 3, version 17, address 0xfec00000, GSI 0-23 0.000000 smpboot: Allowing 2 CPUs, 0 hotplug CPUs 0.000000 Booting paravirtualized kernel on KVM 0.000000 clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns 0.000000 random: get_random_bytes called from start_kernel+0x94/0x486 with crng_init=0 0.000000 setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:2 nr_node_ids:1 0.000000 percpu: Embedded 41 pages/cpu s128600 r8192 d31144 u1048576 0.000000 kvm-stealtime: cpu 0, msr 3f415040 0.000000 PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) 0.000000 Built 1 zonelists, mobility grouping on. Total pages: 255913 0.000000 Kernel command line: reboot=k panic=30 pci=off nomodules console=ttyS0 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd random.trust_cpu=on virtio_mmio.device=4K@0xd0000000:5 virtio_mmio.device=4K@0xd0001000:6 0.000000 PID hash table entries: 4096 (order: 3, 32768 bytes) 0.000000 Memory: 867236K/1039992K available (10252K kernel code, 653K rwdata, 1572K rodata, 1300K init, 2808K bss, 172756K reserved, 0K cma-reserved) 0.000000 SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 0.004000 Hierarchical RCU implementation. 0.004000 RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=2. 0.004000 RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 0.004000 NR_IRQS: 4352, nr_irqs: 56, preallocated irqs: 16 0.004000 Console: colour dummy device 80x25 0.004000 tsc: Detected 2499.998 MHz processor 0.004000 Calibrating delay loop (skipped) preset value.. 4999.99 BogoMIPS (lpj=9999992) 0.004000 pid_max: default: 32768 minimum: 301 0.004000 Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) 0.004274 Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) 0.005000 Mount-cache hash table entries: 2048 (order: 2, 16384 bytes) 0.005703 Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes) 0.006865 Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 0.007436 Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 0.008008 Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization 0.008894 Spectre V2 : Mitigation: Enhanced IBRS 0.009432 Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch 0.010359 Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier 0.011292 Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp 0.012276 Freeing SMP alternatives memory: 32K 0.014864 smpboot: Max logical packages: 2 0.016674 Switched APIC routing to physical x2apic. 0.018377 ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1 0.019094 smpboot: CPU0: Intel(R) Xeon(R) Processor @ 2.50GHz (family: 0x6, model: 0x55, stepping: 0x7) 0.020000 Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only. 0.020000 Hierarchical SRCU implementation. 0.020000 smp: Bringing up secondary CPUs ... 0.020000 x86: Booting SMP configuration: 0.004000 kvm-clock: cpu 1, msr 0:3f7dc041, secondary cpu clock 0.021562 kvm-stealtime: cpu 1, msr 3f515040 0.021727 smpboot: Total of 2 processors activated (9999.99 BogoMIPS) 0.024439 x86/mm: Memory block size: 128MB 0.025140 clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns 0.025185 futex hash table entries: 512 (order: 3, 32768 bytes) 0.026141 NET: Registered protocol family 16 0.029941 HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages 0.032012 HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages 0.034467 pps_core: LinuxPPS API ver. 1 registered 0.035036 pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> 0.036474 dmi: Firmware registration failed. 0.037531 NetLabel: domain hash size = 128 0.038027 NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO 0.038688 NetLabel: unlabeled traffic allowed by default 0.039361 clocksource: Switched to clocksource kvm-clock 0.039994 VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) 0.040120 NET: Registered protocol family 2 0.040794 TCP established hash table entries: 8192 (order: 4, 65536 bytes) 0.041600 TCP bind hash table entries: 8192 (order: 5, 131072 bytes) 0.042346 TCP: Hash tables configured (established 8192 bind 8192) 0.043160 UDP hash table entries: 512 (order: 2, 16384 bytes) 0.043847 UDP-Lite hash table entries: 512 (order: 2, 16384 bytes) 0.044616 NET: Registered protocol family 1 0.045300 RPC: Registered named UNIX socket transport module. 0.045972 RPC: Registered udp transport module. 0.046508 RPC: Registered tcp transport module. 0.047044 RPC: Registered tcp NFSv4.1 backchannel transport module. 0.322064 virtio-mmio: Registering device virtio-mmio.0 at 0xd0000000-0xd0000fff, IRQ 5. 0.323034 virtio-mmio: Registering device virtio-mmio.1 at 0xd0001000-0xd0001fff, IRQ 6. 0.324022 clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x240937b9988, max_idle_ns: 440795218083 ns 0.325156 platform rtc_cmos: registered platform RTC device (no PNP device found) 0.326415 Scanning for low memory corruption every 60 seconds 0.327309 audit: initializing netlink subsys (disabled) 0.328155 Initialise system trusted keyrings 0.329166 audit: type=2000 audit(1622505652.162:1): state=initialized audit_enabled=0 res=1 0.330152 workingset: timestamp_bits=36 max_order=18 bucket_order=0 0.332848 squashfs: version 4.0 (2009/01/31) Phillip Lougher 0.333737 NFS: Registering the id_resolver key type 0.334321 Key type id_resolver registered 0.335272 nfs4filelayout_init: NFSv4 File Layout Driver Registering... 0.338140 Asymmetric key parser 'x509' registered 0.338710 Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) 0.339581 io scheduler noop registered (default) 0.340740 virtio-mmio virtio-mmio.0: Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work. 0.342071 virtio-mmio virtio-mmio.1: Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work. 0.343385 Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled 0.366139 serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a U6_16550A 0.368933 Loading iSCSI transport class v2.0-870. 0.369598 iscsi: registered transport (tcp) 0.370111 tun: Universal TUN/TAP device driver, 1.6 0.370808 hidraw: raw HID events driver (C) Jiri Kosina 0.371500 nf_conntrack version 0.5.0 (8192 buckets, 32768 max) 0.372356 ip_tables: (C) 2000-2006 Netfilter Core Team 0.373174 Initializing XFRM netlink socket 0.373753 NET: Registered protocol family 10 0.375191 NET: Registered protocol family 17 0.376238 Key type dns_resolver registered 0.376828 NET: Registered protocol family 40 0.377867 sched_clock: Marking stable (376783151, 0)->(460833197, -84050046) 0.379384 Loading compiled-in X.509 certificates 0.380634 Loaded X.509 cert 'Build time autogenerated kernel key: c582bb2a1fdabdd195bbc9c8840fff8158852907' 0.381768 zswap: loaded using pool lzo/zbud 0.384240 Freeing unused kernel memory: 1300K 0.400095 Write protecting the kernel read-only data: 14336k 0.402340 Freeing unused kernel memory: 2016K 0.403818 Freeing unused kernel memory: 476K 0.404646 nsm: loading out-of-tree module taints kernel. 0.405293 nsm: module verification failed: signature and/or required key missing - tainting kernel 0.410057 random: kmstool_enclave: uninitialized urandom read (32 bytes read) 0.410884 random: kmstool_enclave: uninitialized urandom read (32 bytes read) sh-4.2$
validate enclave
code:bash
sh-4.2$ nitro-cli describe-enclaves
[
{
"EnclaveID": "i-0736e34244dde3e74-enc179be08ca3b894f",
"ProcessID": 4417,
"EnclaveCID": 16,
"NumberOfCPUs": 2,
"CPUIDs": [
1,
3
],
"MemoryMiB": 512,
"State": "RUNNING",
"Flags": "DEBUG_MODE"
}
]
sh-4.2$ nitro-cli console --enclave-id i-0736e34244dde3e74-enc179be08ca3b894f
Attestation
Nitro Enclaves also supports a cryptographic attestation feature, which allows you to verify an enclave's identity and ensure that only authorized code is running inside it. Attestation ensures that only authorized enclaves are able to decrypt sensitive data and perform specific cryptographic operations
condition keys for AWS KMS key policies that include an enclave's platform configuration registers. This ensures that only authorized enclaves are able to perform cryptographic operations using a specific KMS key.
kmstool-enclave: Enclave内で走るアプリ。Nitro Enclaves SDKでKMS APIを叩き、EC2（Parent instance）から渡された暗号文を複合する
kmstool-instance: EC2（parent instance）で走るアプリ。Enclave内アプリにvsock socketで接続し、aws credentialと暗号文を渡す
kms-key-id: arn:aws:kms:ap-northeast-1:297323088823:key/8001b1d6-892d-446f-ad2a-6f8902dc9b5d
kms key policy
code:json
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::297323088823:role/aws-reserved/sso.amazonaws.com/ap-southeast-1/AWSReservedSSO_AdministratorAccess_a0c742b51c15f716"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Enable decrypt from enclave",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::297323088823:role/test-ec2-role"
},
"Action": "kms:Decrypt",
"Resource": "*",
"Condition": {
"StringEqualsIgnoreCase": {
"kms:RecipientAttestation:ImageSha384": "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"kms:RecipientAttestation:PCR0":"c23f62608bca532ca89537d8e5bdce115fa6279119395fbe1bff6ae1f6ec6882d9606d40d41d59c321631275dc1415ac"
}
}
},
{
"Sid": "Enable encrypt from instance",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::297323088823:role/test-ec2-role"
},
"Action": "kms:Encrypt",
"Resource": "*"
},
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::297323088823:role/test-ec2-role"
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
}
]
}
set up vsock proxy
code:bash
sh-4.2$ vsock-proxy 8000 kms.ap-northeast-1.amazonaws.com 443
sh-4.2$ export AWS_DEFAULT_REGION=ap-northeast-1
sh-4.2$ ENCLAVE_CID=$(nitro-cli describe-enclaves | jq -r .0.EnclaveCID) sh-4.2$ MESSAGE="Hello, KMS\!"
sh-4.2$ KMS_KEY_ARN=arn:aws:kms:ap-northeast-1:297323088823:key/8001b1d6-892d-446f-ad2a-6f8902dc9b5d
sh-4.2$ CIPHERTEXT=$(aws kms encrypt --key-id "$KMS_KEY_ARN" --plaintext "$MESSAGE" --query CiphertextBlob --output text)
sh-4.2$ echo "$CIPHERTEXT" | base64 --decode > /tmp/cipher-text
sh-4.2$ aws kms decrypt --key-id $KMS_KEY_ARN --ciphertext-blob fileb:///tmp/cipher-text
# expected Behavior
An error occurred (AccessDeniedException) when calling the Decrypt operation: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
sh-4.2$
sh-4.2$ docker run --network host -it kmstool-instance /kmstool_instance --cid "$ENCLAVE_CID" --region "$AWS_DEFAULT_REGION" "$CIPHERTEXT"
WARN 2021-06-01T00:20:37Z 00007fcf94e92700 socket - id=0x7fcf88002e20 fd=7: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer. DEBUG 2021-06-01T00:20:37Z 00007fcf94e92700 channel - id=0x7fcf880035b0: no message pool is currently stored in the event-loop local storage, adding 0x7fcf880038b0 with max message size 16384, message count 4, with 4 small blocks of 128 bytes. IQoJb3JpZ2luX2VjEAcaDmFwLW5vcnRoZWFzdC0xIkcwRQIhAJQfAl3YkI9vIpjGRQsrz50QmIX2tjFjrlv+C84tTgSQAiAwTRzL0+1GBvIOws2+jXsDa6byBskkdV6LiJXqMDPcvirHAwix//////////8BEAEaDDI5NzMyMzA4ODgyMyIMsjSDNL4/LP/0PfY7KpsDk5ZqjWlIk/y/0FAJXERWQtxB4kOr3PucwCzQOp5inyT6fWLDEXNrt+Xx2qai/DBKHpWMPMZAM5gKAUYhDWM8JMuoSrAEMr6t1sH2fUGVJM7QApgGf5WGdZuBHSqQtUtCLJZssQ8G4WbQngV91cG26bY91vPkDG19qQ31JbFzUV+1T5mlPV1O8+4Y0byR0IUWQu5CnJEv9ouBXRcPAB+xsNFupkeF60ZWawgDHpEzCi4esrVWx+FTZsKydZtuLNIhHsEj3FzgpNPJUy1piCFmHEWtfGfvMRNir15GnutY8FX3VVTkm4XVNyFb8abLDrVsIyYeI01pHX1IcVn6S8Zk+rkIjaqr7VgpLzcGrRhQqWEbSDx+N/jtG5/6RVg3DFiaI61+tVfHfMbhf7TcgNsKm5KTyb19TPTzRIimcTUa7c60JaDdoPaajvwMatbpS9+KZ+sYOeoybFFog85MFKRix/Ay7sRIJMMOY15O7hhEteViw7brm70zcdKY2NHTJSmUSm6Iiw3JOw5Qtriamzj9zCHRxUFdVkReqqQuMNLj1YUGOusBShtNBUQh5QZBT9MUFHacTJt+BzdyesdPSXOAeR5XZ5dkEzn+j63basDudPVOpLNdFxuL4YG8oh+l38mYYKGb1HbzNrQ/g1pNSwKMKD4UR5XlByOKpycGOvC6A6L+uiQ09ejKgMmbzBOz/qd32qYW3oaYg7WwvnLo+apS5lEhzYkuBSFwmsLHvt/uMvKLlew5w9z6aVqrcQ4I660OX6BA3DHOmgwMZrbVjtom+HqiCLkMByNxkZ+RFtbkSsGGuxR0ULlSYQpphjfJBbew2WEjjheZ2fNkfeKvvO1HeTeuhFhWVqZlF8Xtkn9YHg==
Object = { "Operation": "SetClient", "AwsAccessKeyId": "ASIAUKOOWJO3XQGFZXV3", "AwsSecretAccessKey": "cYfyIQFhjYKYYVL0uICLWYvsqLK3U\/N7PO81jZ2H", "AwsSessionToken": "IQoJb3JpZ2luX2VjEAcaDmFwLW5vcnRoZWFzdC0xIkcwRQIhAJQfAl3YkI9vIpjGRQsrz50QmIX2tjFjrlv+C84tTgSQAiAwTRzL0+1GBvIOws2+jXsDa6byBskkdV6LiJXqMDPcvirHAwix\/\/\/\/\/\/\/\/\/\/8BEAEaDDI5NzMyMzA4ODgyMyIMsjSDNL4\/LP\/0PfY7KpsDk5ZqjWlIk\/y\/0FAJXERWQtxB4kOr3PucwCzQOp5inyT6fWLDEXNrt+Xx2qai\/DBKHpWMPMZAM5gKAUYhDWM8JMuoSrAEMr6t1sH2fUGVJM7QApgGf5WGdZuBHSqQtUtCLJZssQ8G4WbQngV91cG26bY91vPkDG19qQ31JbFzUV+1T5mlPV1O8+4Y0byR0IUWQu5CnJEv9ouBXRcPAB+xsNFupkeF60ZWawgDHpEzCi4esrVWx+FTZsKydZtuLNIhHsEj3FzgpNPJUy1piCFmHEWtfGfvMRNir15GnutY8FX3VVTkm4XVNyFb8abLDrVsIyYeI01pHX1IcVn6S8Zk+rkIjaqr7VgpLzcGrRhQqWEbSDx+N\/jtG5\/6RVg3DFiaI61+tVfHfMbhf7TcgNsKm5KTyb19TPTzRIimcTUa7c60JaDdoPaajvwMatbpS9+KZ+sYOeoybFFog85MFKRix\/Ay7sRIJMMOY15O7hhEteViw7brm70zcdKY2NHTJSmUSm6Iiw3JOw5Qtriamzj9zCHRxUFdVkReqqQuMNLj1YUGOusBShtNBUQh5QZBT9MUFHacTJt+BzdyesdPSXOAeR5XZ5dkEzn+j63basDudPVOpLNdFxuL4YG8oh+l38mYYKGb1HbzNrQ\/g1pNSwKMKD4UR5XlByOKpycGOvC6A6L+uiQ09ejKgMmbzBOz\/qd32qYW3oaYg7WwvnLo+apS5lEhzYkuBSFwmsLHvt\/uMvKLlew5w9z6aVqrcQ4I660OX6BA3DHOmgwMZrbVjtom+HqiCLkMByNxkZ+RFtbkSsGGuxR0ULlSYQpphjfJBbew2WEjjheZ2fNkfeKvvO1HeTeuhFhWVqZlF8Xtkn9YHg==", "AwsRegion": "ap-northeast-1" }
WARN 2021-06-01T00:20:37Z 00007fcf94e92700 socket - id=0x7fcf88017960 fd=7: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer. WARN 2021-06-01T00:20:37Z 00007fcf94e92700 socket - id=0x7fcf88002e20 fd=7: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer. Object = { "Status": "Ok" }
Object = { "Operation": "Decrypt", "Ciphertext": "AQICAHjYtYgyOropB\/glEBRgDVA+1m\/mfZ1RRGH\/7e46K7MIvQH2Lri+GqGaPlWxYTmWbPItAAAAajBoBgkqhkiG9w0BBwagWzBZAgEAMFQGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMYm2dbBqunkkpjmbxAgEQgCezbsli0LPnz1\/QLr5Z06f\/AywAhPpULWugg5e2vly0QJ6\/OJOkrLw=" }
Object = { "Status": "Ok", "Message": "SGVsbG8sIEtNU1wh" }
Hello, KMS\!
sh-4.2$
Log
cloudtrail
Decrypt, GenerateDataKey, GenerateRandom
code:json
{
"eventVersion": "1.08",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAUKOOWJO3R3WYNNPUH:i-0736e34244dde3e74",
"arn": "arn:aws:sts::297323088823:assumed-role/test-ec2-role/i-0736e34244dde3e74",
"accountId": "297323088823",
"accessKeyId": "ASIAUKOOWJO3XQGFZXV3",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AROAUKOOWJO3R3WYNNPUH",
"arn": "arn:aws:iam::297323088823:role/test-ec2-role",
"accountId": "297323088823",
"userName": "test-ec2-role"
},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2021-05-31T23:31:30Z"
},
"ec2RoleDelivery": "2.0"
}
},
"eventTime": "2021-06-01T00:20:37Z",
"eventSource": "kms.amazonaws.com",
"eventName": "Decrypt",
"awsRegion": "ap-northeast-1",
"sourceIPAddress": "54.168.188.252",
"requestParameters": {
"encryptionAlgorithm": "SYMMETRIC_DEFAULT"
},
"responseElements": null,
"additionalEventData": {
"recipient": {
"attestationDocumentModuleId": "i-0736e34244dde3e74-enc0179c4df758b2837",
"attestationDocumentEnclaveImageDigest": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}
},
"requestID": "892a8776-000a-4181-9227-13ae58bdf114",
"eventID": "164c2dec-f98b-48a7-b8b1-09447398f22a",
"readOnly": true,
"resources": [
{
"accountId": "297323088823",
"type": "AWS::KMS::Key",
"ARN": "arn:aws:kms:ap-northeast-1:297323088823:key/8001b1d6-892d-446f-ad2a-6f8902dc9b5d"
}
],
"eventType": "AwsApiCall",
"managementEvent": true,
"eventCategory": "Management",
"recipientAccountId": "297323088823"
}